LIMBO: A Scalable Algorithm to Cluster Categorical Data

Clustering is a problem of great practical importance in numerous applications. The problem of clustering becomes more challenging when the data is categorical, that is, when there is no inherent distance measure between data values. In this work, we introduce LIMBO, a scalable hierarchical categorical clustering algorithm that builds on the Information Bottleneck (IB) framework for quantifying the relevant information preserved when clustering. We use the IB framework to define a distance measure for categorical tuples and we also present a novel distance measure for categorical attribute values. We show how the LIMBO algorithm can be used to cluster both tuples and attribute values. LIMBO handles large data sets by producing a summary model for the data. We propose two different versions of LIMBO, where we either control the size or the accuracy of the model. We present an experimental evaluation of both versions of LIMBO, and we study how clustering quality in information theoretic clustering algorithms compares to other categorical clustering algorithms. LIMBO also supports a tradeoff between efficiency (in terms of space and time). We quantify this trade-off and we demonstrate that LIMBO allows for substantial improvements in efficiency with negligible decrease in quality. LIMBO is a hierarchical algorithm that produces clusterings for a range of k values (where k is the number of clusters). We take advantage of this feature to examine heuristics for selecting good clusterings (with natural values of k) within this range. 1